SherbyNet - /page/wikibook/entry.mu

<!DOCTYPE html>

<title>Sparse PCA</title>

</head>

Sparse PCA

</h1>

</header>

Sparse principal component analysis (SPCA or sparse PCA) is a technique used in statistical analysis and, in particular, in the analysis of <a href="Multivariate_analysis" class="mw-redirect" title="Multivariate analysis">multivariate</a> data sets. It extends the classic method of <a href="Principal_component_analysis" title="Principal component analysis">principal component analysis</a> (PCA) for the reduction of dimensionality of data by introducing sparsity structures to the input variables.

A particular disadvantage of ordinary PCA is that the principal components are usually linear combinations of all input variables. SPCA overcomes this disadvantage by finding components that are linear combinations of just a few input variables (SPCs). This means that some of the coefficients of the linear combinations defining the SPCs, called loadings,<a href="#cite_note-1">[note 1]</a> are equal to zero. The number of nonzero loadings is called the cardinality of the SPC.

<div class="mw-heading mw-heading2"><h2 id="Mathematical_formulation">Mathematical formulation</h2></div>

Consider a data <a href="Matrix_(mathematics)" title="Matrix (mathematics)">matrix</a>, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X}</annotation>

</semantics>

</math><img src="./68baa052181f707c662844a465bfeeb135e82bab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\displaystyle X}" loading="lazy">, where each of the <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy"> columns represent an input variable, and each of the <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy"> rows represents an independent sample from data population. One assumes each column of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X}</annotation>

</semantics>

</math><img src="./68baa052181f707c662844a465bfeeb135e82bab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\displaystyle X}" loading="lazy"> has mean zero, otherwise one can subtract column-wise mean from each element of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X}</annotation>

</semantics>

</math><img src="./68baa052181f707c662844a465bfeeb135e82bab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\displaystyle X}" loading="lazy">.

Let <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Sigma ={\frac {1}{n-1}}X^{\top }X}">

<mfrac>

<mrow>

</mrow>

</mfrac>

</mrow>

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Sigma ={\frac {1}{n-1}}X^{\top }X}</annotation>

</semantics>

</math><img src="./8fd354178263354a8853f1a8c87d64aba6112785.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.005ex; width:16.498ex; height:5.343ex;" alt="{\displaystyle \Sigma ={\frac {1}{n-1}}X^{\top }X}" loading="lazy"> be the empirical <a href="Covariance_matrix" title="Covariance matrix">covariance matrix</a> of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X}</annotation>

</semantics>

</math><img src="./68baa052181f707c662844a465bfeeb135e82bab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\displaystyle X}" loading="lazy">, which has dimension <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p\times p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p\times p}</annotation>

</semantics>

</math><img src="./67e6de741a3aa8176f5a487de8e8f602aa75c5e4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:5.269ex; height:2.009ex;" alt="{\displaystyle p\times p}" loading="lazy">.

Given an integer <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle k}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle k}</annotation>

</semantics>

</math><img src="./c3c9a2c7b599b37105512c5d570edc034056dd40.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.211ex; height:2.176ex;" alt="{\displaystyle k}" loading="lazy"> with <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle 1\leq k\leq p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle 1\leq k\leq p}</annotation>

</semantics>

</math><img src="./2356b80db7fdb9c72394f696a6b696610d75104c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:9.74ex; height:2.509ex;" alt="{\displaystyle 1\leq k\leq p}" loading="lazy">, the sparse PCA problem can be formulated as maximizing the variance along a direction represented by vector <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle v\in \mathbb {R} ^{p}}">

<msup>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle v\in \mathbb {R} ^{p}}</annotation>

</semantics>

</math><img src="./5211e2a7d81c7c652e972e5d7c42699e0d5bbbd3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.706ex; height:2.343ex;" alt="{\displaystyle v\in \mathbb {R} ^{p}}" loading="lazy"> while constraining its cardinality:

<mtr>

<mtd>

</mtd>

<mtd>

<msup>

</mrow>

</msup>

</mtd>

</mtr>

<mtr>

<mtd>

<mtext>subject to</mtext>

</mrow>

</mtd>

<mtd>

<msub>

<mrow>

</mrow>

</mrow>

</msub>

</mtd>

</mtr>

<mtr>

<mtd>

<msub>

<mrow>

</mrow>

</mrow>

</msub>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}\max \quad &v^{T}\Sigma v\\{\text{subject to}}\quad &\left\Vert v\right\Vert _{2}=1\\&\left\Vert v\right\Vert _{0}\leq k.\end{aligned}}}</annotation>

</semantics>

</math><img src="./43e43fd583a7f4f0f857a75ada6adbcaeb40aefd.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -4.171ex; width:22.368ex; height:9.509ex;" alt="{\displaystyle {\begin{aligned}\max \quad &v^{T}\Sigma v\\{\text{subject to}}\quad &\left\Vert v\right\Vert _{2}=1\\&\left\Vert v\right\Vert _{0}\leq k.\end{aligned}}}" loading="lazy">Eq. 1</dd></dl>

The first constraint specifies that v is a unit vector. In the second constraint, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \left\Vert v\right\Vert _{0}}">

<msub>

<mrow>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \left\Vert v\right\Vert _{0}}</annotation>

</semantics>

</math><img src="./1bbb8d8b94f4d8c2f12e72938565c1124a875170.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:4.507ex; height:3.009ex;" alt="{\displaystyle \left\Vert v\right\Vert _{0}}" loading="lazy"> represents the <a href="L0_norm" class="mw-redirect" title="L0 norm"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \ell _{0}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \ell _{0}}</annotation>

</semantics>

</math><img src="./d18f7cb79dd41b63d6aca9ec6b957c225a0aea81.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.024ex; height:2.509ex;" alt="{\displaystyle \ell _{0}}" loading="lazy"> pseudo-norm</a> of v, which is defined as the number of its non-zero components. So the second constraint specifies that the number of non-zero components in v is less than or equal to k, which is typically an integer that is much smaller than dimension p. The optimal value of <a href="#math_Eq._1">Eq. 1</a> is known as the k-sparse largest <a href="Eigenvalue" class="mw-redirect" title="Eigenvalue">eigenvalue</a>.

If one takes k=p, the problem reduces to the ordinary <a href="Principal_component_analysis" title="Principal component analysis">PCA</a>, and the optimal value becomes the largest eigenvalue of covariance matrix Σ.

After finding the optimal solution v, one deflates Σ to obtain a new matrix

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Sigma _{1}=\Sigma -(v^{T}\Sigma v)vv^{T},}</annotation>

</semantics>

</math><img src="./c3efb0a1fdfdaff97f1c9d3e7489b951935eadf4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:21.772ex; height:3.176ex;" alt="{\displaystyle \Sigma _{1}=\Sigma -(v^{T}\Sigma v)vv^{T},}" loading="lazy"></dd></dl>

and iterate this process to obtain further principal components. However, unlike PCA, sparse PCA cannot guarantee that different principal components are <a href="Orthogonal" class="mw-redirect" title="Orthogonal">orthogonal</a>. In order to achieve orthogonality, additional constraints must be enforced.

The following equivalent definition is in matrix form.

Let <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle V}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle V}</annotation>

</semantics>

</math><img src="./af0f6064540e84211d0ffe4dac72098adfa52845.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.787ex; height:2.176ex;" alt="{\displaystyle V}" loading="lazy"> be a p×p symmetric matrix, one can rewrite the sparse PCA problem as

<mtr>

<mtd>

</mtd>

<mtd>

</mtd>

</mtr>

<mtr>

<mtd>

<mtext>subject to</mtext>

</mrow>

</mtd>

<mtd>

</mtd>

</mtr>

<mtr>

<mtd>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mtd>

</mtr>

<mtr>

<mtd>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}\max \quad &Tr(\Sigma V)\\{\text{subject to}}\quad &Tr(V)=1\\&\Vert V\Vert _{0}\leq k^{2}\\&Rank(V)=1,V\succeq 0.\end{aligned}}}</annotation>

</semantics>

</math><img src="./c75c28b22c9fd44c17ffa9f38254a24a57dfc05d.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -5.671ex; width:34.091ex; height:12.509ex;" alt="{\displaystyle {\begin{aligned}\max \quad &Tr(\Sigma V)\\{\text{subject to}}\quad &Tr(V)=1\\&\Vert V\Vert _{0}\leq k^{2}\\&Rank(V)=1,V\succeq 0.\end{aligned}}}" loading="lazy">Eq. 2</dd></dl>

Tr is the <a href="Matrix_trace" class="mw-redirect" title="Matrix trace">matrix trace</a>, and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \Vert V\Vert _{0}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \Vert V\Vert _{0}}</annotation>

</semantics>

</math><img src="./4a0c3bd31ecf220db6ca3d0a90ba4087d2a55b25.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:5.166ex; height:2.843ex;" alt="{\displaystyle \Vert V\Vert _{0}}" loading="lazy"> represents the non-zero elements in matrix V.

The last line specifies that V has <a href="Matrix_rank" class="mw-redirect" title="Matrix rank">matrix rank</a> one and is <a href="Positive_semidefinite_matrix" class="mw-redirect" title="Positive semidefinite matrix">positive semidefinite</a>.

The last line means that one has <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle V=vv^{T}}">

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle V=vv^{T}}</annotation>

</semantics>

</math><img src="./d69a46973fb2768a1892ecb906b6986a8c9a2497.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:8.53ex; height:2.676ex;" alt="{\displaystyle V=vv^{T}}" loading="lazy">, so <a href="#math_Eq._2">Eq. 2</a> is equivalent to <a href="#math_Eq._1">Eq. 1</a>.

Moreover, the rank constraint in this formulation is actually redundant, and therefore sparse PCA can be cast as the following mixed-integer semidefinite program<a href="#cite_note-BCJ20-2">[1]</a>

<mtr>

<mtd>

</mtd>

<mtd>

</mtd>

</mtr>

<mtr>

<mtd>

<mtext>subject to</mtext>

</mrow>

</mtd>

<mtd>

</mtd>

</mtr>

<mtr>

<mtd>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<mfrac>

</mfrac>

</mrow>

<msub>

</mrow>

</msub>

</mtd>

</mtr>

<mtr>

<mtd>

<msup>

</mrow>

</msup>

</mrow>

</munder>

<msub>

</mrow>

</msub>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}\max \quad &Tr(\Sigma V)\\{\text{subject to}}\quad &Tr(V)=1\\&\vert V_{i,i}\vert \leq z_{i},\forall i\in \{1,...,p\},\vert V_{i,j}\vert \leq {\frac {1}{2}}z_{i},\forall i,j\in \{1,...,p\}:i\neq j,\\&V\succeq 0,z\in \{0,1\}^{p},\sum _{i}z_{i}\leq k\end{aligned}}}</annotation>

</semantics>

</math><img src="./2a052ba9d91b8b40384c9a4b79692cfddca557b6.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -8.005ex; width:75.98ex; height:17.176ex;" alt="{\displaystyle {\begin{aligned}\max \quad &Tr(\Sigma V)\\{\text{subject to}}\quad &Tr(V)=1\\&\vert V_{i,i}\vert \leq z_{i},\forall i\in \{1,...,p\},\vert V_{i,j}\vert \leq {\frac {1}{2}}z_{i},\forall i,j\in \{1,...,p\}:i\neq j,\\&V\succeq 0,z\in \{0,1\}^{p},\sum _{i}z_{i}\leq k\end{aligned}}}" loading="lazy">Eq. 3</dd></dl>

Because of the cardinality constraint, the maximization problem is hard to solve exactly, especially when dimension p is high. In fact, the sparse PCA problem in <a href="#math_Eq._1">Eq. 1</a> is <a href="NP-hard" class="mw-redirect" title="NP-hard">NP-hard</a> in the strong sense.<a href="#cite_note-TP14-3">[2]</a>

<div class="mw-heading mw-heading2"><h2 id="Computational_considerations">Computational considerations</h2></div>

As most sparse problems, variable selection in SPCA is a computationally intractable non-convex NP-hard problem,<a href="#cite_note-:1-4">[3]</a> therefore greedy sub-optimal algorithms are often employed to find solutions.

Note also that SPCA introduces hyperparameters quantifying in what capacity large parameter values are penalized.<a href="#cite_note-5">[4]</a> These might need <a href="Hyperparameter_optimization" title="Hyperparameter optimization">tuning</a> to achieve satisfactory performance, thereby adding to the total computational cost.

<div class="mw-heading mw-heading2"><h2 id="Algorithms_for_SPCA">Algorithms for SPCA</h2></div>

Several alternative approaches (of <a href="#math_Eq._1">Eq. 1</a>) have been proposed, including

<ul><li>a regression framework,<a href="#cite_note-6">[5]</a></li>

<li>a penalized matrix decomposition framework,<a href="#cite_note-7">[6]</a></li>

<li>a convex relaxation/semidefinite programming framework,<a href="#cite_note-SDP-8">[7]</a></li>

<li>a generalized power method framework<a href="#cite_note-9">[8]</a></li>

<li>an alternating maximization framework<a href="#cite_note-10">[9]</a></li>

<li>forward-backward greedy search and exact methods using branch-and-bound techniques,<a href="#cite_note-11">[10]</a></li>

<li>a certifiably optimal branch-and-bound approach<a href="#cite_note-12">[11]</a></li>

<li>Bayesian formulation framework.<a href="#cite_note-13">[12]</a></li>

<li>A certifiably optimal mixed-integer semidefinite branch-and-cut approach <a href="#cite_note-BCJ20-2">[1]</a></li></ul>

The methodological and theoretical developments of Sparse PCA as well as its applications in scientific studies are recently reviewed in a survey paper.<a href="#cite_note-14">[13]</a>

<div class="mw-heading mw-heading3"><h3 id="Notes_on_Semidefinite_Programming_Relaxation">Notes on Semidefinite Programming Relaxation</h3></div>

It has been proposed that sparse PCA can be approximated by <a href="Semidefinite_programming" title="Semidefinite programming">semidefinite programming</a> (SDP).<a href="#cite_note-SDP-8">[7]</a> If one drops the rank constraint and relaxes the cardinality constraint by a 1-norm <a href="Convex_set" title="Convex set">convex</a> constraint, one gets a semidefinite programming relaxation, which can be solved efficiently in polynomial time:

<mtr>

<mtd>

</mtd>

<mtd>

</mtd>

</mtr>

<mtr>

<mtd>

<mtext>subject to</mtext>

</mrow>

</mtd>

<mtd>

</mtd>

</mtr>

<mtr>

<mtd>

<msup>

</mrow>

</mrow>

</msup>

</mrow>

</mrow>

</mrow>

</mtd>

</mtr>

<mtr>

<mtd>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}\max \quad &Tr(\Sigma V)\\{\text{subject to}}\quad &Tr(V)=1\\&\mathbf {1} ^{T}|V|\mathbf {1} \leq k\\&V\succeq 0.\end{aligned}}}</annotation>

</semantics>

</math><img src="./2865937d68be33b4fa4f042cfffbce8009d28358.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -5.671ex; width:24.358ex; height:12.509ex;" alt="{\displaystyle {\begin{aligned}\max \quad &Tr(\Sigma V)\\{\text{subject to}}\quad &Tr(V)=1\\&\mathbf {1} ^{T}|V|\mathbf {1} \leq k\\&V\succeq 0.\end{aligned}}}" loading="lazy">Eq. 3</dd></dl>

In the second constraint, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {1} }">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathbf {1} }</annotation>

</semantics>

</math><img src="./235ffc0f1788b720aef5caa7b97246a84421fd0e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.337ex; height:2.176ex;" alt="{\displaystyle \mathbf {1} }" loading="lazy"> is a p×1 vector of ones, and |V| is the matrix whose elements are the absolute values of the elements of V.

The optimal solution <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle V}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle V}</annotation>

</semantics>

</math><img src="./af0f6064540e84211d0ffe4dac72098adfa52845.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.787ex; height:2.176ex;" alt="{\displaystyle V}" loading="lazy"> to the relaxed problem <a href="#math_Eq._3">Eq. 3</a> is not guaranteed to have rank one. In that case, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle V}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle V}</annotation>

</semantics>

</math><img src="./af0f6064540e84211d0ffe4dac72098adfa52845.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.787ex; height:2.176ex;" alt="{\displaystyle V}" loading="lazy"> can be truncated to retain only the dominant eigenvector.

While the semidefinite program does not scale beyond n=300 covariates, it has been shown that a second-order cone relaxation of the semidefinite relaxation is almost as tight and successfully solves problems with n=1000s of covariates <a href="#cite_note-15">[14]</a>

<div class="mw-heading mw-heading2"><h2 id="Applications">Applications</h2></div>

<div class="mw-heading mw-heading3"><h3 id="Financial_Data_Analysis">Financial Data Analysis</h3></div>

Suppose ordinary PCA is applied to a dataset where each input variable represents a different asset, it may generate principal components that are weighted combination of all the assets. In contrast, sparse PCA would produce principal components that are weighted combination of only a few input assets, so one can easily interpret its meaning. Furthermore, if one uses a trading strategy based on these principal components, fewer assets imply less transaction costs.

<div class="mw-heading mw-heading3"><h3 id="Biology">Biology</h3></div>

Consider a dataset where each input variable corresponds to a specific gene. Sparse PCA can produce a principal component that involves only a few genes, so researchers can focus on these specific genes for further analysis.

<div class="mw-heading mw-heading3"><h3 id="High-dimensional_Hypothesis_Testing">High-dimensional Hypothesis Testing</h3></div>

Contemporary datasets often have the number of input variables (<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p}</annotation>

</semantics>

</math><img src="./81eac1e205430d1f40810df36a0edffdc367af36.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:1.259ex; height:2.009ex;" alt="{\displaystyle p}" loading="lazy">) comparable with or even much larger than the number of samples (<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n}</annotation>

</semantics>

</math><img src="./a601995d55609f2d9f5e233e36fbe9ea26011b3b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.395ex; height:1.676ex;" alt="{\displaystyle n}" loading="lazy">). It has been shown that if <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p/n}">

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p/n}</annotation>

</semantics>

</math><img src="./3265bb7382222c10e1b6f77bf9fd781b2ebf83dc.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; margin-left: -0.089ex; width:3.816ex; height:2.843ex;" alt="{\displaystyle p/n}" loading="lazy"> does not converge to zero, the classical PCA is not <a href="Consistency_(statistics)" title="Consistency (statistics)">consistent</a>. In other words, if we let <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle k=p}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle k=p}</annotation>

</semantics>

</math><img src="./ce9e3515dc907812a8d3b1c6bd035977194cce80.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:5.479ex; height:2.509ex;" alt="{\displaystyle k=p}" loading="lazy"> in <a href="#math_Eq._1">Eq. 1</a>, then

the optimal value does not converge to the largest eigenvalue of data population when the sample size <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle n\rightarrow \infty }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle n\rightarrow \infty }</annotation>

</semantics>

</math><img src="./9702f04f2d0e5b887b99faeeffb0c4cfd8263eee.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:7.333ex; height:1.843ex;" alt="{\displaystyle n\rightarrow \infty }" loading="lazy">, and the optimal solution does not converge to the direction of maximum variance.

But sparse PCA can retain consistency even if <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle p\gg n.}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle p\gg n.}</annotation>

</semantics>

</math><img src="./2a3c9d4c990f0883b9aa7b1dd2c8dd0bec2404e2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; margin-left: -0.089ex; width:6.914ex; height:2.176ex;" alt="{\displaystyle p\gg n.}" loading="lazy">

The k-sparse largest eigenvalue (the optimal value of <a href="#math_Eq._1">Eq. 1</a>) can be used to discriminate an isometric model, where every direction has the same variance, from a spiked covariance model in high-dimensional setting.<a href="#cite_note-16">[15]</a> Consider a hypothesis test where the null hypothesis specifies that data <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X}</annotation>

</semantics>

</math><img src="./68baa052181f707c662844a465bfeeb135e82bab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\displaystyle X}" loading="lazy"> are generated from a multivariate normal distribution with mean 0 and covariance equal to an identity matrix, and the alternative hypothesis specifies that data <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle X}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle X}</annotation>

</semantics>

</math><img src="./68baa052181f707c662844a465bfeeb135e82bab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.98ex; height:2.176ex;" alt="{\displaystyle X}" loading="lazy"> is generated from a spiked model with signal strength <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \theta }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \theta }</annotation>

</semantics>

</math><img src="./6e5ab2664b422d53eb0c7df3b87e1360d75ad9af.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.09ex; height:2.176ex;" alt="{\displaystyle \theta }" loading="lazy">:

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle H_{0}:X\sim N(0,I_{p}),\quad H_{1}:X\sim N(0,I_{p}+\theta vv^{T}),}</annotation>

</semantics>

</math><img src="./91d621d140b778ef746a0f903a1b541b50f0fe8c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:47.884ex; height:3.343ex;" alt="{\displaystyle H_{0}:X\sim N(0,I_{p}),\quad H_{1}:X\sim N(0,I_{p}+\theta vv^{T}),}" loading="lazy"></dd></dl>

where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle v\in \mathbb {R} ^{p}}">

<msup>

</mrow>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle v\in \mathbb {R} ^{p}}</annotation>

</semantics>

</math><img src="./5211e2a7d81c7c652e972e5d7c42699e0d5bbbd3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:6.706ex; height:2.343ex;" alt="{\displaystyle v\in \mathbb {R} ^{p}}" loading="lazy"> has only k non-zero coordinates. The largest k-sparse eigenvalue can discriminate the two hypotheses if and only if <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \theta >\Theta ({\sqrt {k\log(p)/n}})}">

<msqrt>

</mrow>

</msqrt>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \theta >\Theta ({\sqrt {k\log(p)/n}})}</annotation>

</semantics>

</math><img src="./733f3451dc11b0e690d6c505f2accad3578957ba.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.838ex; width:20.236ex; height:4.843ex;" alt="{\displaystyle \theta >\Theta ({\sqrt {k\log(p)/n}})}" loading="lazy">.

Since computing k-sparse eigenvalue is NP-hard, one can approximate it by the optimal value of semidefinite programming relaxation (<a href="#math_Eq._3">Eq. 3</a>). If that case, we can discriminate the two hypotheses if <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \theta >\Theta ({\sqrt {k^{2}\log(p)/n}})}">

<msqrt>

<msup>

</mrow>

</msup>

</mrow>

</msqrt>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \theta >\Theta ({\sqrt {k^{2}\log(p)/n}})}</annotation>

</semantics>

</math><img src="./1407bb125a86a69d6b8d8cbd11c45c9f0c8b3915.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.671ex; width:21.29ex; height:4.843ex;" alt="{\displaystyle \theta >\Theta ({\sqrt {k^{2}\log(p)/n}})}" loading="lazy">. The additional <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\sqrt {k}}}">

<msqrt>

</msqrt>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\sqrt {k}}}</annotation>

</semantics>

</math><img src="./de05da7afd02cfd22d059bf17a0c165e3079d5d4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.147ex; height:3.009ex;" alt="{\displaystyle {\sqrt {k}}}" loading="lazy"> term cannot be improved by any other polynomial time algorithm if the <a href="Planted_clique" title="Planted clique">planted clique conjecture</a> holds.

<div class="mw-heading mw-heading2"><h2 id="Software/source_code">Software/source code</h2></div>

<ul><li>amanpg - R package for Sparse PCA using the Alternating Manifold Proximal Gradient Method <a href="#cite_note-17">[16]</a></li>

<li>elasticnet – R package for Sparse Estimation and Sparse PCA using Elastic-Nets <a href="#cite_note-18">[17]</a></li>

<li>epca – R package for exploratory principal component analysis for large-scale dataset, including sparse principal component analysis and sparse matrix approximation. <a href="#cite_note-19">[18]</a></li>

<li>nsprcomp - R package for sparse and/or non-negative PCA based on thresholded power iterations<a href="#cite_note-20">[19]</a></li>

<li><a href="Scikit-learn" title="Scikit-learn">scikit-learn</a> – Python library for machine learning which contains Sparse PCA and other techniques in the decomposition module.<a href="#cite_note-21">[20]</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="Notes">Notes</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist">

<li id="cite_note-1"><a href="#cite_ref-1">^</a> The term loadings is inappropriately used for these vectors, which should be called coefficients, instead. The naming comes from the term loadings used in factor analysis to design the values generating the common covariance matrix. Since in standard PCA the loadings are equal to the coefficients, the term loadings has been used for the coefficients. This is quite unfortunate, because in SPCA the coefficients are not equal to the loadings.

</li>

</ol></div></div>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite id="CITEREFDimitris_BertsimasRyan_Cory-WrightJean_Pauphilet2020" class="citation arxiv cs1">Dimitris Bertsimas; Ryan Cory-Wright; Jean Pauphilet (2020). "Solving Large-Scale Sparse PCA to Certifiable (Near) Optimality". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2005.05195">2005.05195</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/math.OC">math.OC</a>].</cite>

</li>

<cite id="CITEREFAndreas_M._TillmannMarc_E._Pfetsch2013" class="citation journal cs1">Andreas M. Tillmann; Marc E. Pfetsch (2013). "The Computational Complexity of the Restricted Isometry Property, the Nullspace Property, and Related Concepts in Compressed Sensing". <a href="IEEE_Transactions_on_Information_Theory" title="IEEE Transactions on Information Theory">IEEE Transactions on Information Theory</a>. 60 (2): 1248–1259. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1205.2081">1205.2081</a>. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.760.2559">10.1.1.760.2559</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FTIT.2013.2290112">10.1109/TIT.2013.2290112</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:2788088">2788088</a>.</cite>

</li>

<li id="cite_note-:1-4"><a href="#cite_ref-:1_4-0">^</a> <cite id="CITEREFMoghaddamWeissAvidan2007" class="citation book cs1">Moghaddam, Baback; Weiss, Yair; Avidan, Shai (2007-09-07). Schölkopf, Bernhard; Platt, John; Hofmann, Thomas (eds.). <a rel="nofollow" class="external text" href="https://direct.mit.edu/books/book/3168/Advances-in-Neural-Information-Processing-Systems">Advances in Neural Information Processing Systems 19: Proceedings of the 2006 Conference</a>. The MIT Press. pp. 915–922. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.7551%2Fmitpress%2F7503.001.0001">10.7551/mitpress/7503.001.0001</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-0-262-25691-9</bdi>.</cite>

</li>

<li id="cite_note-5"><a href="#cite_ref-5">^</a> <cite id="CITEREFZouand_Tibshirani2006" class="citation journal cs1">Zou, Hui; and Tibshirani, Robert (2006-06-01). <a rel="nofollow" class="external text" href="https://www.tandfonline.com/doi/abs/10.1198/106186006X113430">"Sparse Principal Component Analysis"</a>. Journal of Computational and Graphical Statistics. 15 (2): 265–286. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1198%2F106186006X113430">10.1198/106186006X113430</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1061-8600">1061-8600</a>.</cite> <code class="cs1-code">{{cite journal}}</code>: <code class="cs1-code">|first2=</code> missing <code class="cs1-code">|last2=</code> (help)CS1 maint: multiple names: authors list (link)

</li>

<cite id="CITEREFHui_ZouTrevor_HastieRobert_Tibshirani2006" class="citation journal cs1">Hui Zou; Trevor Hastie; Robert Tibshirani (2006). <a rel="nofollow" class="external text" href="http://www-stat.stanford.edu/~hastie/Papers/spc_jcgs.pdf">"Sparse principal component analysis"</a> (PDF). <a href="Journal_of_Computational_and_Graphical_Statistics" title="Journal of Computational and Graphical Statistics">Journal of Computational and Graphical Statistics</a>. 15 (2): 262–286. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.62.580">10.1.1.62.580</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1198%2F106186006x113430">10.1198/106186006x113430</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:5730904">5730904</a>.</cite>

</li>

<cite id="CITEREFFan_ChenKarl_Rohe2021" class="citation journal cs1">Fan Chen; Karl Rohe (2021). <a rel="nofollow" class="external text" href="https://www.tandfonline.com/eprint/8VKDCFHDPTQHXB47MXIR/full?target=10.1080/10618600.2023.2256502">"A New Basis for Sparse Principal Component Analysis"</a>. <a href="Journal_of_Computational_and_Graphical_Statistics" title="Journal of Computational and Graphical Statistics">Journal of Computational and Graphical Statistics</a>. 33 (2): 421–434. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2007.00596">2007.00596</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1080%2F10618600.2023.2256502">10.1080/10618600.2023.2256502</a>.</cite>

</li>

<cite id="CITEREFAlexandre_d’AspremontLaurent_El_GhaouiMichael_I._JordanGert_R._G._Lanckriet2007" class="citation journal cs1">Alexandre d’Aspremont; Laurent El Ghaoui; Michael I. Jordan; Gert R. G. Lanckriet (2007). <a rel="nofollow" class="external text" href="http://www.cmap.polytechnique.fr/~aspremon/PDF/sparsesvd.pdf">"A Direct Formulation for Sparse PCA Using Semidefinite Programming"</a> (PDF). <a href="SIAM_Review" class="mw-redirect" title="SIAM Review">SIAM Review</a>. 49 (3): 434–448. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/cs/0406021">cs/0406021</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1137%2F050645506">10.1137/050645506</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:5490061">5490061</a>.</cite>

</li>

<cite id="CITEREFMichel_JourneeYurii_NesterovPeter_RichtarikRodolphe_Sepulchre2010" class="citation journal cs1">Michel Journee; Yurii Nesterov; Peter Richtarik; Rodolphe Sepulchre (2010). <a rel="nofollow" class="external text" href="http://jmlr.csail.mit.edu/papers/volume11/journee10a/journee10a.pdf">"Generalized Power Method for Sparse Principal Component Analysis"</a> (PDF). <a href="Journal_of_Machine_Learning_Research" title="Journal of Machine Learning Research">Journal of Machine Learning Research</a>. 11: 517–553. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/0811.4724">0811.4724</a>. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2008arXiv0811.4724J">2008arXiv0811.4724J</a>. CORE Discussion Paper 2008/70.</cite>

</li>

<cite id="CITEREFPeter_RichtarikMajid_JahaniS._Damla_AhipasaogluMartin_Takac2021" class="citation journal cs1">Peter Richtarik; Majid Jahani; S. Damla Ahipasaoglu; Martin Takac (2021). <a rel="nofollow" class="external text" href="https://link.springer.com/article/10.1007/s11081-020-09562-3">"Alternating Maximization: Unifying Framework for 8 Sparse PCA Formulations and Efficient Parallel Codes"</a>. Optimization and Engineering. 22 (3): 1493–1519. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1212.4137">1212.4137</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2Fs11081-020-09562-3">10.1007/s11081-020-09562-3</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:2549610">2549610</a>.</cite>

</li>

<cite id="CITEREFBaback_MoghaddamYair_WeissShai_Avidan2005" class="citation conference cs1">Baback Moghaddam; Yair Weiss; Shai Avidan (2005). <a rel="nofollow" class="external text" href="http://books.nips.cc/papers/files/nips18/NIPS2005_0643.pdf">"Spectral Bounds for Sparse PCA: Exact and Greedy Algorithms"</a> (PDF). Advances in Neural Information Processing Systems. Vol. 18. MIT Press.</cite>

</li>

<cite id="CITEREFLauren_BerkDimitris_Bertsimas2019" class="citation journal cs1">Lauren Berk; Dimitris Bertsimas (2019). "Certifiably optimal sparse principal component analysis". <a href="Mathematical_Programming_Computation" class="mw-redirect" title="Mathematical Programming Computation">Mathematical Programming Computation</a>. 11 (3). Springer: 381–420. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2Fs12532-018-0153-6">10.1007/s12532-018-0153-6</a>. <a href="Hdl_(identifier)" class="mw-redirect" title="Hdl (identifier)">hdl</a>:<a rel="nofollow" class="external text" href="https://hdl.handle.net/1721.1%2F131566">1721.1/131566</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:126998398">126998398</a>.</cite>

</li>

<cite id="CITEREFYue_GuanJennifer_Dy2009" class="citation journal cs1">Yue Guan; <a href="Jennifer_Dy" title="Jennifer Dy">Jennifer Dy</a> (2009). <a rel="nofollow" class="external text" href="http://jmlr.csail.mit.edu/proceedings/papers/v5/guan09a/guan09a.pdf">"Sparse Probabilistic Principal Component Analysis"</a> (PDF). Journal of Machine Learning Research Workshop and Conference Proceedings. 5: 185.</cite>

</li>

<cite id="CITEREFHui_ZouLingzhou_Xue2018" class="citation journal cs1">Hui Zou; Lingzhou Xue (2018). <a rel="nofollow" class="external text" href="https://doi.org/10.1109%2Fjproc.2018.2846588">"A Selective Overview of Sparse Principal Component Analysis"</a>. <a href="Proceedings_of_the_IEEE" title="Proceedings of the IEEE">Proceedings of the IEEE</a>. 106 (8): 1311–1320. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2Fjproc.2018.2846588">10.1109/jproc.2018.2846588</a>.</cite>

</li>

<cite id="CITEREFDimitris_BertsimasRyan_Cory-Wright2020" class="citation journal cs1">Dimitris Bertsimas; Ryan Cory-Wright (2020). <a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.orl.2019.12.003">"On polyhedral and second-order cone decompositions of semidefinite optimization problems"</a>. <a href="Operations_Research_Letters" title="Operations Research Letters">Operations Research Letters</a>. 48 (1). Elsevier: 78–85. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1910.03143">1910.03143</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.orl.2019.12.003">10.1016/j.orl.2019.12.003</a>.</cite>

</li>

<cite id="CITEREFQuentin_BerthetPhilippe_Rigollet2013" class="citation journal cs1">Quentin Berthet; Philippe Rigollet (2013). "Optimal Detection of Sparse Principal Components in High Dimension". <a href="Annals_of_Statistics" title="Annals of Statistics">Annals of Statistics</a>. 41 (1): 1780–1815. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1202.5070">1202.5070</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1214%2F13-aos1127">10.1214/13-aos1127</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:7162068">7162068</a>.</cite>

</li>

<li id="cite_note-17"><a href="#cite_ref-17">^</a> <a rel="nofollow" class="external autonumber" href="https://cran.r-project.org/web/packages/amanpg/index.html">[1]</a> <a rel="nofollow" class="external free" href="https://cran.r-project.org/web/packages/amanpg/index.html">https://cran.r-project.org/web/packages/amanpg/index.html</a>

</li>

<li id="cite_note-18"><a href="#cite_ref-18">^</a> <a rel="nofollow" class="external autonumber" href="https://cran.r-project.org/web/packages/elasticnet/index.html">[2]</a> <a rel="nofollow" class="external free" href="https://cran.r-project.org/web/packages/elasticnet/index.html">https://cran.r-project.org/web/packages/elasticnet/index.html</a>

</li>

<li id="cite_note-19"><a href="#cite_ref-19">^</a> <a rel="nofollow" class="external autonumber" href="https://cran.r-project.org/web/packages/epca/index.html">[3]</a> <a rel="nofollow" class="external free" href="https://cran.r-project.org/web/packages/epca/index.html">https://cran.r-project.org/web/packages/epca/index.html</a>

</li>

<li id="cite_note-20"><a href="#cite_ref-20">^</a> <a rel="nofollow" class="external autonumber" href="https://cran.r-project.org/web/packages/nsprcomp/index.html">[4]</a> <a rel="nofollow" class="external free" href="https://cran.r-project.org/web/packages/nsprcomp/index.html">https://cran.r-project.org/web/packages/nsprcomp/index.html</a>

</li>

<li id="cite_note-21"><a href="#cite_ref-21">^</a> <a rel="nofollow" class="external autonumber" href="http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.SparsePCA.html">[5]</a> <a rel="nofollow" class="external free" href="http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.SparsePCA.html">http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.SparsePCA.html</a>

</li>

</ol></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-07-22" href="https://en.wikipedia.org/wiki/?title=Sparse_PCA&oldid=1301908312">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>